Generally, the “cloud” refers to computing resources, such as software, services, and information, connected via a network, e.g., the Internet. “Cloud computing” can thus be defined as Internet-based computing in which these computing resources are dynamically provisioned and allocated to client or user computers or other devices on-demand from a pool of resources available via the cloud. Cloud computing is envisioned as a way to allocate access to these resources according to particular needs of a user. One application of cloud computing is to process multi-phase, distributed computing jobs. In these types of applications, the data is divided up and processed in parallel by multiple tasks during each execution phase, thereby reducing the total amount of time to complete the job.